A Human-Robot Collaborative Reinforcement Learning Algorithm

This paper presents a new reinforcement learning algorithm that enables collaborative learning between a robot and a human. The algorithm which is based on the Q(λ) approach expedites the learning process by taking advantage of human intelligence and expertise. The algorithm denoted as CQ(λ) provides the robot with self awareness to adaptively switch its collaboration level from autonomous (self performing, the robot decides which actions to take, according to its learning function) to semi-autonomous (a human advisor guides the robot and the robot combines this knowledge into its learning function). This awareness is represented by a self test of its learning performance. The approach of variable autonomy is demonstrated and evaluated using a fixed-arm robot for finding the optimal shaking policy to empty the contents of a plastic bag. A comparison between the CQ(λ) and This work was partially supported by the Paul Ivanier Center for Robotics Research and Production Management, and by the Rabbi W. Gunther Plaut Chair in Manufacturing Engineering, Ben-Gurion University of the Negev. We would like to thank Dr. Amir Shapiro from the Department of Mechanical Engineering for his suggestions, and comments. We would also like to thank Dr. Sigal Berman and Mr. Amit Gil from the Department of Industrial Engineering and Management for their critical reading of the text. U. Kartoun (B) Microsoft Medical Medial Lab (M3L), Microsoft, Washington DC, USA e-mail: [email protected] H. Stern · Y. Edan Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel H. Stern e-mail: [email protected] Y. Edan e-mail: [email protected] 218 J Intell Robot Syst (2010) 60:217–239 the traditional Q(λ)-reinforcement learning algorithm, resulted in faster convergence for the CQ(λ) collaborative reinforcement learning algorithm.